Re: Re: Need urgent help with Solr spatial search using SpatialRecursivePrefixTreeFieldType

2019-10-01 Thread David Smiley
Do you know how URLs are structured? They include name=value pairs separated by ampersands. This takes precedence over the contents of any particular name or value. Consequently looking at your parenthesis doesn't make sense since the open-close span ampersands and thus go to different filter

Re: filter in JSON Query DSL

2019-10-01 Thread Mikhail Khludnev
Raised https://issues.apache.org/jira/browse/SOLR-13808. Thanks, Jochen! On Mon, Sep 30, 2019 at 4:26 PM Mikhail Khludnev wrote: > Jochen, right! Sorry for didn't get your point earlier. {!bool filter=} > means Lucene filter, not Solr's one. I suppose {!bool cache=true} flag can > be easily

Re: Solrcloud export all results sorted by score

2019-10-01 Thread Walter Underwood
I had to do this recently on a Solr Cloud cluster. I wanted to export all the IDs, but they weren’t stored as docvalues. The fastest approach was to fetch all the IDs in one request. First, I make a request for zero rows to get the numFound. Then I fetch numFound+1000 (in case docs were added

Re: Solrcloud export all results sorted by score

2019-10-01 Thread Erick Erickson
First, thanks for taking the time to ask a question with enough supporting details that I can hope to be able to answer in one exchange ;). It’s a pleasure to see. Second, NP with asking on Stack Overflow, they have some excellent answers there. But you’re right, this list gets more

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Erick Erickson
bq. My customer requested me to achieve 1000 qps with a single Solr. You need to talk to your customer then and adjust expectations. You say you get 60 QPS with 40% CPU utilization. So let’s say QPS is perfectly linear to CPU utilization and you can get CPU utilization to run at 100%. In that

Using policy/preference/... to keep certain collections split?

2019-10-01 Thread Koen De Groote
Hello, I'm trying to achieve that no replicas of collection A exist on the same host as a replica of collection B. Using Solr7.6 Both are large collections/will become large collections and load on 1 should not impact the other in any way. They will both be queried heavily, simultaneously. So,

Re: Re: Need urgent help with Solr spatial search using SpatialRecursivePrefixTreeFieldType

2019-10-01 Thread anushka gupta
Thanks, Could you please help me in combining two geofilt fqs as the following gives error, it treats ")" as part of the d parameter and gives error that 'd=80)' is not a valid param: ({!geofilt}=adminLatLon=33.0198431,-96.6988856=80)+OR+({!geofilt}=adminLatLon=50.2171726,8.265894=80) --

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Toke Eskildsen
On Tue, 2019-10-01 at 19:08 +0900, Yasufumi Mizoguchi wrote: > * The number of Firing hosts : 6 > * [Each host]ThreadGroup.num_threads : 200 > * [Each host]ThreadGroup.ramp_time : 600 > * [Each host]ThreadGroup.duration: 1800 > * [Each host]ThreadGroup.delay: 0 > * The number of sample queries:

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Yasufumi Mizoguchi
Hi Paras, Thank you for your advice. I will confirm JMeter's settings in addition to JVM options. Yes, we are using documentCache and *after* finishing load test, we will comment that out. (By customer requests, we cannot update cache settings till the test ends...) Thanks, Yasufumi

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Yasufumi Mizoguchi
It is difficult to answer that for me. My customer requested me to achieve 1000 qps with single Solr. Thanks, Yasufumi. 2019年10月1日(火) 14:59 Jörn Franke : > Why do you need 1000 qps? > > > > Am 30.09.2019 um 07:45 schrieb Yasufumi Mizoguchi < > yasufumi0...@gmail.com>: > > > > Hi, > > > > I am

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Paras Lehana
Hi Yasufumi, Followings are current load test set up. Did you try decreasing ramp_time and increasing num_threads or Firing hosts? Out of 1800 secs, you are giving 200 threads a maximum of 600 seconds to get ready. I don't increase with this value when I want to test parallel requests. In your

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Yasufumi Mizoguchi
Thank you for replying me. I will try to resize NewRatio. Thanks, Yasufumi. 2019年10月1日(火) 11:19 Deepak Goel : > Hello > > Can you please try increasing 'new size' and 'max new size' to 1GB+? > > Deepak > > On Mon, 30 Sep 2019, 13:35 Yasufumi Mizoguchi, > wrote: > > > Hi, Deepak. > > Thank you

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Yasufumi Mizoguchi
Thank you for your response. Now, we have no JVM monitoring. But checking GC logs, I found no major GC during load test. As you saying, heap size might be too large and I am planning to reduce that. Thanks, Yasufumi 2019年9月30日(月) 23:19 Walter Underwood : > 31G is still a very large heap. We

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Yasufumi Mizoguchi
Ah, sorry. Not JUnit, we use JMeter. Thanks, Yasufumi 2019年10月1日(火) 19:08 Yasufumi Mizoguchi : > Thank you for replying me. > > Followings are current load test set up. > > * Load test program : JUnit > * The number of Firing hosts : 6 > * [Each host]ThreadGroup.num_threads : 200 > * [Each

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Yasufumi Mizoguchi
Thank you for your response. Our index has about 25,000,000 docs. Is this quite difficult to achieve 1000 qps with single Solr instance even if all index files are cached on OS page cache? And as you pointed out, there are lots of minor GC during the test, I will try to resize NewRatio. Thanks,

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Yasufumi Mizoguchi
Thank you for replying me. Followings are current load test set up. * Load test program : JUnit * The number of Firing hosts : 6 * [Each host]ThreadGroup.num_threads : 200 * [Each host]ThreadGroup.ramp_time : 600 * [Each host]ThreadGroup.duration: 1800 * [Each host]ThreadGroup.delay: 0 * The

Re: SOlr Spatial Search - Filter and sort on multiple cities

2019-10-01 Thread anushka gupta
Thanks, Could you please also let me know how to combine two geofilt fqs because if i use '&' like : admin_directory_search_geolocation?q=david=({!geofilt=adminLatLon=33.0198431,-96.6988856=80})+OR+({!geofilt=adminLatLon=50.2171726,8.265894=80}) then it gives me error as it takes ")" paranthesis

Solrcloud export all results sorted by score

2019-10-01 Thread Edward Turner
Hi all, As far as I understand, SolrCloud currently does not allow the use of sorting by the pseudofield, score in the /export request handler (i.e., get the results in relevancy order). If we do attempt this, we get an exception, "org.apache.solr.search.SyntaxError: Scoring is not currently

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Paras Lehana
Besides what all have suggested, can you share your testing setup? Are you using JMeter? I'm asking this to confirm that your setup is "actually trying" to generate 1000 simultaneous threads and not bottlenecking the process. I get 170 qps easily on a simpler server but I remember that I could not

Re: Throughput does not increase in spite of low CPU usage

2019-10-01 Thread Jörn Franke
Why do you need 1000 qps? > Am 30.09.2019 um 07:45 schrieb Yasufumi Mizoguchi : > > Hi, > > I am trying some tests to confirm if single Solr instance can perform over > 1000 queries per second(!). > > But now, although CPU usage is 40% or so and iowait is almost 0%, > throughput does not