Re: Getting totalTermFreq and docFreq for terms

2017-02-22 Thread Joel Bernstein
The idea of adding a terms.ttf parameter sounds fine to me. And It would be good to get terms.list better integrated into the TermsComponent. In general I think it's time for more attention to be paid to the TermsComponent. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Feb 22, 2017 at

Re: Getting totalTermFreq and docFreq for terms

2017-02-22 Thread Shai Erera
Hmm .. so if I want to add totalTermFreq to the response, it will break the current output format of TermsComponent, which returns for each term only the docFreq. What's our BWC policy for such API and is there a way to handle it? I can add a new terms.ttf parameter, and so if you set it to true,

Re: Getting totalTermFreq and docFreq for terms

2017-02-22 Thread Shai Erera
Looks like this could be a very easy addition to TermsComponent? From what I read in the code, it uses TermContext to compute/hold the stats, and the latter already has docFreq and totalTermFreq (!!). It's just that TermsComponent does not output TTF (only computes it...): for(int i=0;

Re: Getting totalTermFreq and docFreq for terms

2017-02-22 Thread Joel Bernstein
Yeah, I think expanding the functionality of the terms component looks like the right place to add these stats. I plan on exposing these types of terms stats as Streaming Expression functions but I would likely use the terms component under the covers. Joel Bernstein

Re: Getting totalTermFreq and docFreq for terms

2017-02-22 Thread Shai Erera
No, they are not global distributed stats. I am willing to live with approximated stats though (unless again, there's an API which can give me both). I wonder why doesn't Terms component return ttf in addition to docfreq. The API (at the Lucene level) is right there already. On Wed, Feb 22, 2017

Re: Getting totalTermFreq and docFreq for terms

2017-02-22 Thread Joel Bernstein
Hi Shai, Do ttf and docfreq return global stats in distributed mode? I wasn't aware that there was a mechanism for aggregating values in the field list. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Feb 22, 2017 at 7:18 AM, Shai Erera wrote: > Hi > > I am currently

Getting totalTermFreq and docFreq for terms

2017-02-22 Thread Shai Erera
Hi I am currently using function queries to obtain these two statistics, as I didn't see a better or more explicit API and the Terms component only returns docFreq, but not totalTermFreq. The way I use the API is submit requests as follows: curl "