[
https://issues.apache.org/jira/browse/SOLR-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15377292#comment-15377292
]
Shalin Shekhar Mangar commented on SOLR-9290:
---------------------------------------------
Hmm, you're right Yonik. But we've always had an idle timeout for the http
connector in jetty set to 50 seconds (I traced this back to SOLR-128). So after
50 seconds of inactivity, Jetty closes that connection from its end and the
client's socket goes to CLOSE_WAIT state. As you said, this connection cannot
be re-used anymore. When httpclient tries to use the connection, it does the
stale check, sees the CLOSE_WAIT state and terminates the connection and gives
a new one to Solr.
So all the connections that suddenly do not show up in CLOSE_WAIT and we
assumed that they went to ESTABLISHED state were actually terminated.
So in summary, our assumption that connections in CLOSE_WAIT are kept around
because of re-use is wrong but it still doesn't change the solution that I've
proposed. We could also think of increasing the value of Jetty's idle timeout
as a separate change but the idle eviction thread would still be necessary.
> TCP-connections in CLOSE_WAIT spikes during heavy indexing when SSL is enabled
> ------------------------------------------------------------------------------
>
> Key: SOLR-9290
> URL: https://issues.apache.org/jira/browse/SOLR-9290
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 5.5.1, 5.5.2
> Reporter: Anshum Gupta
> Priority: Critical
> Attachments: SOLR-9290-debug.patch, SOLR-9290-debug.patch, index.sh,
> setup-solr.sh, setup-solr.sh
>
>
> Heavy indexing on Solr with SSL leads to a lot of connections in CLOSE_WAIT
> state.
> At my workplace, we have seen this issue only with 5.5.1 and could not
> reproduce it with 5.4.1 but from my conversation with Shalin, he knows of
> users with 5.3.1 running into this issue too.
> Here's an excerpt from the email [~shaie] sent to the mailing list (about
> what we see:
> {quote}
> 1) It consistently reproduces on 5.5.1, but *does not* reproduce on 5.4.1
> 2) It does not reproduce when SSL is disabled
> 3) Restarting the Solr process (sometimes both need to be restarted), the
> count drops to 0, but if indexing continues, they climb up again
> When it does happen, Solr seems stuck. The leader cannot talk to the
> replica, or vice versa, the replica is usually put in DOWN state and
> there's no way to fix it besides restarting the JVM.
> {quote}
> Here's the mail thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201607.mbox/%3c46cc66220a8143dc903fa34e79205...@vp-exc01.dips.local%3E
> Creating this issue so we could track this and have more people comment on
> what they see.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
