First, please don’t use the “schemaless” mode (add-unknown-fields-to-schema in
your solrconfig) while load testing. There’s quite a bit of work being done
when Solr discovers an unknown field that’ll cause some instability under heavy
load.
Second, when you put a large batch against Solr, you have the possibility of
the update just taking longer than the timeout. There are several timeouts you
can increase, see the “solr.xml” section of the ref guide.
Best,
Erick
> On Nov 19, 2019, at 12:29 PM, Fiz N <fiznewy...@gmail.com> wrote:
>
>
> Hello Solr Experts,
>
> Just wanted to follow up in case my question.Would appreciate help on this.
>
> SOLR Version : 6.6.2
> OS – Linux 3.1.2
> JDK – 1.8
>
> Shard – 16 – All are active.
> Xms – 16 gb
> Xmx – 16 gb
> Host has 64 cores.
>
> the update processor chain.attaching the complete updateRequestProcessorChain
> in a file.
> Attaching physical and CPU memory screenshot.
>
> There are multiple threads sending products to solr. tested with batch size
> per thread 50 & 100, it worked fine without error and if the batch size is
> 1000 the following error occurs.
>
>
> I am getting the following error when the batch size is 1000. Please advise.
>
> 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard15_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15} -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> 2 Async exceptions during distributed update:
> 10.YYY.40.62:8983 failed to respond
> 10.YYY.40.62:8983 failed to respond
>
> 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard7_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7} -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: 10.YYY.40.81:8983 failed to
> respond
>
>
> 2019-11-14T19:36:11,599 - ERROR
> [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
> http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2 r:core_node26
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131] -
> {ConcurrentUpdateSolrClient.url=http://10.YYY.40.68:8983/solr/ducts_shard11_replica2,
> collection=c:ducts, core=x:ducts_shard3_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3} -
> error
> org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
> ~[httpclient-4.4.1.jar:4.4.1]
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> ~[httpclient-4.4.1.jar:4.4.1]
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
> ~[httpcore-4.4.1.jar:4.4.1]
>
>
> 2019-11-14T19:36:14,567 - ERROR
> [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
> http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1 r:core_node25
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
> -
> {ConcurrentUpdateSolrClient.url=http://10.YYY.40.62:8983/solr/ducts_shard2_replica1,
> collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11} -
> error
> java.net.SocketException: Broken pipe (Write failed)
> at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[?:1.8.0_232]
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> ~[?:1.8.0_232]
> at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> ~[?:1.8.0_232]
>
>
> 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
> {collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11} -
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: Broken pipe (Write failed)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>
>
> Thanks
>
> <sample.txt>
