You might be interested in:
https://issues.apache.org/jira/browse/SOLR-13389
Kevin Risden
On Sun, May 5, 2019 at 1:35 PM Gus Heck <gus.h...@gmail.com> wrote:
> I'm working with a client that's trying to process a lot of data (billions
> of docs) via a streaming expression, and the initial query is (not
> surprisingly) taking a long time. Lots of various types of timeouts have
> been cropping up and I've found myself thinking I solved some only to
> discover that the settings in solr.xml are far less wide reaching than I
> thought initially. The present 5% scale cluster seems to hit one particular
> time out about 50% of the time which has made it particularly confusing.
> I'm guessing it's probably depending on something like how busy the
> virtualization in Amazon is, just barely making it when it gets more
> resources and timing out if anything is starved.
>
> As I look around the code base I'm finding a LOT of places where timeouts
> on SolrClients and CloudSolrClients are just arbitrarily set to one-off
> constant values. The one bugging me right now is
>
> public abstract class SolrClientBuilder<B extends SolrClientBuilder<B>> {
>
> protected HttpClient httpClient;
> protected ResponseParser responseParser;
> protected Integer connectionTimeoutMillis = 15000;
> protected Integer socketTimeoutMillis = 120000;
>
> Which I am unable to change because of this code in SolrStream:
>
> /**
> * Opens the stream to a single Solr instance.
> **/
> public void open() throws IOException {
> if(cache == null) {
> client = new HttpSolrClient.Builder(baseUrl).build();
> } else {
> client = cache.getHttpSolrClient(baseUrl);
> }
>
> I need to make this particular case configurable, so that I can get
> results from a very long running query, but I sense that there is a much
> wider problem in that we don't seem to have any organized plan for how
> socket timeouts are set/managed in the code.
>
> What thoughts have people had on this front?
>
> -Gus
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
