t; Sent: Tuesday, March 19, 2019 3:29:17 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Indexing is CPU bound. If you have enough RAM, SSD disks, and enough client
> threads, you should be able to drive CPU to over 90%.
>
> Start with tw
time.
I will try with Solr Could cluster, maybe get better speed there.
//Aaron
From: Walter Underwood
Sent: Tuesday, March 19, 2019 3:29:17 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
Indexing is CPU bound. If you have enough RAM
happening,
>> almost half of the request has such long delay. The more document added in
>> one request the more delay it has.
>>
>>
>> From: Emir Arnautović
>> Sent: Tuesday, March 19, 2019 12:30:33 PM
>> To: solr-u
I'll second Emir's suggestion to try disabling swap. "I doubt swap would
affect it since there is such huge free memory." -- sounds reasonable, but
has not been my experience, and the stats you sent indicate that swap is in
fact being used. Also, note that in many cases setting vm.swappiness=0 is
Indexing is CPU bound. If you have enough RAM, SSD disks, and enough client
threads, you should be able to drive CPU to over 90%.
Start with two client threads per CPU. That allows one thread to be sending
data over the network while another is waiting for Solr to process the batch.
A couple
Isn't there somthing about largePageTables which must be enabled
in JAVA and also supported by OS for such huge heaps?
Just a guess.
Am 19.03.19 um 15:01 schrieb Jörn Franke:
It could be an issue with jdk 8 that may not be suitable for such large heaps.
Have more nodes with smaller heaps (eg
It could be an issue with jdk 8 that may not be suitable for such large heaps.
Have more nodes with smaller heaps (eg 31 gb)
> Am 18.03.2019 um 11:47 schrieb Aaron Yingcai Sun :
>
> Hello, Solr!
>
>
> We are having some performance issue when try to send documents for solr to
> index. The
> From: Emir Arnautović
> Sent: Tuesday, March 19, 2019 1:00:19 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> If you start indexing with just a single thread/client, do you still see
> slow bulks?
>
> Emir
> --
> Moni
n Considerations. indexed fields. The number of indexed fields
greatly increases the following: Memory usage during indexing ; Segment merge
time
From: Emir Arnautović
Sent: Tuesday, March 19, 2019 1:00:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr ind
m: Emir Arnautović
> Sent: Tuesday, March 19, 2019 12:30:33 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Just to add different perspective here: how do you send documents to Solr?
> Are those log lines from your client? Maybe it is not Solr that is
:33 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
Just to add different perspective here: how do you send documents to Solr? Are
those log lines from your client? Maybe it is not Solr that is slow. Could it
be network or client itself. If you have some dry run on client
her faster way to index such big amount of data?
>
>
> BRs
>
> //Aaron
>
>
> From: Walter Underwood
> Sent: Monday, March 18, 2019 4:59:20 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Solr is not designed to have cons
028-189979 DBG1:doc_count: 10 , doc_size: 584 KB, Res code:
> 200, QTime: 22800 ms, Request time: 22802 ms.
> 190318-162821.056-189948 DBG1:doc_count: 10 , doc_size: 670 KB, Res code:
> 200, QTime: 34193 ms, Request time: 34195 ms.
> 190318-162821.062-189983 DBG1:doc_count: 10 , doc_s
028-189979 DBG1:doc_count: 10 , doc_size: 584 KB, Res code:
> 200, QTime: 22800 ms, Request time: 22802 ms.
> 190318-162821.056-189948 DBG1:doc_count: 10 , doc_size: 670 KB, Res code:
> 200, QTime: 34193 ms, Request time: 34195 ms.
> 190318-162821.062-189983 DBG1:doc_count: 10 , doc_size: 67
Sent: Monday, March 18, 2019 2:54:25 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
One other thing to look at besides the heap is your commit settings. We've
experienced something similar, and changing commit settings alleviated the
issue.
Are you opening a search on ever
: Emir Arnautović
> Sent: Monday, March 18, 2019 2:19:19 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Hi Aaron,
> Without looking too much into numbers, my bet would be that it is large
> heap that is causing issues. I would decrease is
___
> From: Emir Arnautović
> Sent: Monday, March 18, 2019 2:19:19 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solr index slow response
>
> Hi Aaron,
> Without looking too much into numbers, my bet would be that it is large heap
> that is causi
G1:doc_count: 10 , doc_size: 605 KB, Res code:
> 200, QTime: 108 ms, Request time: 110 ms.
> 190318-142655.304-160208 DBG1:doc_count: 10 , doc_size: 481 KB, Res code:
> 200, QTime: 89 ms, Request time: 90 ms.
> 190318-142655.410-160208 DBG1:doc_count: 10 , doc_size: 468 KB, Res code:
&
time.
BRs
//Aaron
From: Emir Arnautović
Sent: Monday, March 18, 2019 2:19:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr index slow response
Hi Aaron,
Without looking too much into numbers, my bet would be that it is large heap
that is causing issues
he.org
Subject: Re: Solr index slow response
On Mon, 2019-03-18 at 10:47 +, Aaron Yingcai Sun wrote:
> Solr server is running on a quit powerful server, 32 cpus, 400GB RAM,
> while 300 GB is reserved for solr, [...]
300GB for Solr sounds excessive.
> Our application send 100 docume
path":"...",
>> "commandLineArgs":["-Xms100G",
>> "-Xmx300G",
>> "-DSTOP.PORT=8079",
>> "-DSTOP.KEY=..",
>> "-Dsolr.solr.home=..",
>> "-Djett
-Djetty.port=8983"],
> "startTime":"2019-03-18T09:35:27.892Z",
> "upTimeMS":9258422}},
> "system":{
>"name":"Linux",
>"arch":"amd64",
>"availableProcessors":32,
&
On Mon, 2019-03-18 at 10:47 +, Aaron Yingcai Sun wrote:
> Solr server is running on a quit powerful server, 32 cpus, 400GB RAM,
> while 300 GB is reserved for solr, [...]
300GB for Solr sounds excessive.
> Our application send 100 documents to solr per request, json encoded.
> the size is
quot;:14.72,
"version":"3.0.101-311.g08a8a9d-default",
"committedVirtualMemorySize":2547960700928,
"freePhysicalMemorySize":4530696192,
"freeSwapSpaceSize":3486846976,
"processCpuLoad":0.3257436126790475,
"p
Hi Aaron,
Which version of Solr? How did you configure your heap? Is it standalone Solr
or SolrCloud? A single server? Do you use some monitoring tool? Do you see some
spikes, pauses or CPU usage is constant?
Thanks,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr &
Hello, Solr!
We are having some performance issue when try to send documents for solr to
index. The repose time is very slow and unpredictable some time.
Solr server is running on a quit powerful server, 32 cpus, 400GB RAM, while 300
GB is reserved for solr, while this happening, cpu usage
26 matches
Mail list logo
