@Martin, yeah I expected that to work initially, but it has no effect on
the streaming expression code path.
@Kevin, thx for the link :)
On Sun, May 5, 2019, 2:49 PM Martin Gainty <mgai...@hotmail.com> wrote:
> both connectionTimeout and socketTimeout are declared in
> $SOLR_HOME/solr.xml
>
> <shardHandlerFactory name="shardHandlerFactory"
> class="HttpShardHandlerFactory">
> <int name="socketTimeout">${socketTimeout:600000}</int>
> <int name="connTimeout">${connTimeout:60000}</int>
> </shardHandlerFactory>
>
> </solr>
>
>
> ------------------------------
> *From:* Gus Heck <gus.h...@gmail.com>
> *Sent:* Sunday, May 5, 2019 1:35 PM
> *To:* dev
> *Subject:* Socket Timeouts
>
> I'm working with a client that's trying to process a lot of data (billions
> of docs) via a streaming expression, and the initial query is (not
> surprisingly) taking a long time. Lots of various types of timeouts have
> been cropping up and I've found myself thinking I solved some only to
> discover that the settings in solr.xml are far less wide reaching than I
> thought initially. The present 5% scale cluster seems to hit one particular
> time out about 50% of the time which has made it particularly confusing.
> I'm guessing it's probably depending on something like how busy the
> virtualization in Amazon is, just barely making it when it gets more
> resources and timing out if anything is starved.
>
> As I look around the code base I'm finding a LOT of places where timeouts
> on SolrClients and CloudSolrClients are just arbitrarily set to one-off
> constant values. The one bugging me right now is
>
> public abstract class SolrClientBuilder<B extends SolrClientBuilder<B>> {
>
> protected HttpClient httpClient;
> protected ResponseParser responseParser;
> protected Integer connectionTimeoutMillis = 15000;
> protected Integer socketTimeoutMillis = 120000;
>
> Which I am unable to change because of this code in SolrStream:
>
> /**
> * Opens the stream to a single Solr instance.
> **/
> public void open() throws IOException {
> if(cache == null) {
> client = new HttpSolrClient.Builder(baseUrl).build();
> } else {
> client = cache.getHttpSolrClient(baseUrl);
> }
>
> I need to make this particular case configurable, so that I can get
> results from a very long running query, but I sense that there is a much
> wider problem in that we don't seem to have any organized plan for how
> socket timeouts are set/managed in the code.
>
> What thoughts have people had on this front?
>
> -Gus
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
