[
https://issues.apache.org/jira/browse/SOLR-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375885#comment-15375885
]
Hoss Man commented on SOLR-9290:
--------------------------------
Somebody sanity check my understanding / summary description of the root
issue...
* Solr's use of HttpClient for intra-node communication has historically always
had the potential to result in connections sitting "idle" (ie: in a CLOSE_WAIT
state) for possible re-use later -- but these connections are kept open
indefinitely.
** For reasons I don't understand, 'idle' connections are more likely to
(exist? | be kept around indefinitely?) when the intra-node communication is
over SSL.
* {{maxUpdateConnections}} and {{maxUpdateConnectionsPerHost}} have always set
hard upper limits on the number of connections that could ever be created --
let alone in sitting idle in a CLOSE_WAIT state.
* Prior to SOLR-8533, the default values for these limits was relatively low,
making it unlikely that users could ever observe an extreme # of idle /
CLOSE_WAIT threads -- you were more likely to have your Solr cluster crash from
deadlocks then notice any serious OS level problem with too many idle
connections
* After SOLR-8533, the increased default values of these limits made the
problem much more noticeable
* SOLR-4509's changes included use of a new option which results in a
background thread checking for an existing idle connections on the master branch
* This issue address the problem for branch_6x (and older) branches via a
similar background thread
> TCP-connections in CLOSE_WAIT spikes during heavy indexing when SSL is enabled
> ------------------------------------------------------------------------------
>
> Key: SOLR-9290
> URL: https://issues.apache.org/jira/browse/SOLR-9290
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 5.5.1, 5.5.2
> Reporter: Anshum Gupta
> Priority: Critical
> Attachments: SOLR-9290-debug.patch, SOLR-9290-debug.patch,
> setup-solr.sh
>
>
> Heavy indexing on Solr with SSL leads to a lot of connections in CLOSE_WAIT
> state.
> At my workplace, we have seen this issue only with 5.5.1 and could not
> reproduce it with 5.4.1 but from my conversation with Shalin, he knows of
> users with 5.3.1 running into this issue too.
> Here's an excerpt from the email [~shaie] sent to the mailing list (about
> what we see:
> {quote}
> 1) It consistently reproduces on 5.5.1, but *does not* reproduce on 5.4.1
> 2) It does not reproduce when SSL is disabled
> 3) Restarting the Solr process (sometimes both need to be restarted), the
> count drops to 0, but if indexing continues, they climb up again
> When it does happen, Solr seems stuck. The leader cannot talk to the
> replica, or vice versa, the replica is usually put in DOWN state and
> there's no way to fix it besides restarting the JVM.
> {quote}
> Here's the mail thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201607.mbox/%[email protected]%3E
> Creating this issue so we could track this and have more people comment on
> what they see.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
