Hi Christine, Thanks a lot for the posts. Very impressive information (article as well as the youtube video!)
Thanks a lot Merry Xmas and and Happy New Year! Cheers, Arturas On Thu, Dec 24, 2020 at 11:03 AM Christine Poerschke (BLOOMBERG/ LONDON) < cpoersc...@bloomberg.net> wrote: > Hello again Arturas. > > I meant to reply before but somehow lost track of it ... The "Lifecycle of > a Solr Search Request" slides [1] and/or talk [2] may be of interest to you. > > Regards, > Christine > > [1] https://home.apache.org/~hossman/rev2017/ > [2] https://youtu.be/qItRilJLj5o > > From: solr-user@lucene.apache.org At: 12/10/20 21:42:19To: > solr-user@lucene.apache.org > Subject: Re: increasing number of threads for faceting in JSON format > > Hi Christine Munendra et al, > > Wow, you dag into the code and checked weather threads are being blown in > range and term queries! I wish one day to be able to do the same myself. > > How does one get to the level, so one can check the code herself? Is there > like a nice primer or crash course, solr 101 so to say, things you did not > learn in school about solr, but you wish you had learned web page? Well, > I'll take this opportunity to scroll through the lines in the github. Your > answer is very helpful. > > Cheers, > Arturas > > On Thu, Dec 10, 2020 at 7:08 PM Munendra S N <sn.munendr...@gmail.com> > wrote: > > > Thank you Christine. > > Yeah, JSON facet does not support specifying threads. > > > > > > On Thu, Dec 10, 2020, 11:15 PM Christine Poerschke (BLOOMBERG/ LONDON) < > > cpoersc...@bloomberg.net> wrote: > > > > > Hello Arturas and Munendra! > > > > > > In the "Currently, JSON facets have support for specifying the number > of > > > threads." sentence, I wonder if perhaps a "does not" got inadvertently > > > omitted i.e. "Currently, JSON facets does not have support for > specifying > > > the number of threads." was intended? > > > > > > Let me share what I learnt from digging into the code: > > > > > > * "facet.threads" is for field value faceting [1] [2] but you're > > > interested in (JSON) field range faceting as well as JSON field value > > > faceting. > > > > > > * The area of the code [3] that does the JSON field range faceting > shows > > > no obvious threading or parallelisation. > > > > > > Hope that helps? > > > > > > Regards, > > > > > > Christine > > > > > > [1] > > > > > > > https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-para > meters > > > [2] > > > > > > > https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/ > src/java/org/apache/solr/request/SimpleFacets.java > <https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/request/SimpleFacets.java> > > > [3] > > > > > > > https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/ > src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113 > <https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113> > > > > > > From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To: > > > solr-user@lucene.apache.org > > > Subject: Re: increasing number of threads for faceting in JSON format > > > > > > Hi Munedra, > > > > > > This is great that I can get things faster by reducing the gap and by > > > increasing the number of threads. How to reduce gaps I know: one can > > > replace "gap": "+1HOUR" with "gap": "+1MONTH" What should I > > change > > > in the text below to increase the number of threads from one to 20? > > > > > > Cheers, > > > Arturas > > > > > > On Thu, Dec 3, 2020 at 1:54 PM Munendra S N <sn.munendr...@gmail.com> > > > wrote: > > > > > > > Hi, > > > > > > > > Currently, JSON facets have support for specifying the number of > > threads. > > > > In the above request, the range facet is computed over 2 years with a > > gap > > > > of 1 hour. By reducing the number of buckets, computation should > become > > > > much faster > > > > > > > > Regards, > > > > Munendra S N > > > > > > > > > > > > > > > > On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika <maze...@gmail.com> > > > wrote: > > > > > > > > > Hi Solr-Users, > > > > > > > > > > I am trying to better understand the solr capabilities, how one can > > > > > formulate queries in JSON format as well as tweak parameters. > > > Currently I > > > > > have a logs collection (ca 6GB large) with a dozen of attributes > > > running > > > > in > > > > > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd > start > > -h > > > > > localhost -p 5555 -m 4g) > > > > > > > > > > I am playing with faceting functionality in solr and query a couple > > of > > > > > attributes there. My typical query is: > > > > > > > > > > GET http://localhost:5555/solr/db/query > > > > > <http://arteika:5555/solr/logan/query> HTTP/1.1 > > > > > content-type: application/json > > > > > > > > > > { > > > > > "query" : "*:*", > > > > > "limit" : 0, > > > > > "facet": { > > > > > "t" : { > > > > > "type": "terms", > > > > > "field": "fcomp", > > > > > "sort": "index", > > > > > > > > > > "facet": { > > > > > "t_buckets": { > > > > > "type": "range", > > > > > "field": "t", > > > > > "sort": { "t": "asc" }, > > > > > "start": "2018-05-02T17:00:00.000Z", > > > > > "end": "2020-11-16T21:00:00.000Z", > > > > > "gap": "+1HOUR" > > > > > } > > > > > } > > > > > }, > > > > > } > > > > > } > > > > > > > > > > not surprisingly, it takes a bit to compute the result, so I tried > to > > > > > increase the number of threads. How do I do it in JSON format? I > > tried > > > > > adding > > > > > > > > > > { > > > > > "params": { > > > > > "facet.threads": 8 > > > > > }, > > > > > "query" : "*:*", > > > > > ... > > > > > } > > > > > > > > > > and checked the jstack <pid> of the solr java process, but I still > > see > > > > only > > > > > one thread working. Can I configure params through the params > > section? > > > > > > > > > > I also tried > > > > > > > > > > { > > > > > "query" : "*:*", > > > > > "limit" : 0, > > > > > "facet": { > > > > > "t" : { > > > > > "type": "terms", > > > > > "field": "fcomp", > > > > > "sort": "index", > > > > > > > > > > "facet": { > > > > > "t_buckets": { > > > > > "type": "range", > > > > > "field": "t", > > > > > "sort": { "t": "asc" }, > > > > > "start": "2018-05-02T17:00:00.000Z", > > > > > "end": "2020-11-16T21:00:00.000Z", > > > > > "gap": "+1HOUR" > > > > > } > > > > > }, > > > > > "threads":8 > > > > > }, > > > > > } > > > > > } > > > > > > > > > > but this ran in one thread as well. Can I influence the number of > > > threads > > > > > in the "facet" section of JSON? > > > > > > > > > > Cheers, > > > > > Arturas > > > > > > > > > > > > > > > > > > > > > > >
