On Wed, 2020-10-21 at 00:25 +0530, Arunan Sugunakumar wrote:
> Hi Oleg,
> 
> I was able to write a simple Test Server that reproduces this issue.
> I changed the NHttpFileServer example to come up with an example. If
> you send a POST request with a dummy payload and a "Connection:close"
> header to this server in 8443 port, you can see that the connection
> will not be closed (JDK11 & JDK8_261) and the CPU will be spinning
> continuously processing Write Events. If you start the same server in
> JDK8 (JDK8_251), the session will close normally.
> 
> Please go through the example and provide your suggestions.
> 
> Regards,
> Arunan
> 

Hi Arunan

I was able to reproduce the issue with Oracle Java 11.0.8 which is a
reasonably up to date Java 11 runtime. However at the moment I do not
know whether this is a bug in Oracle's TLSv1.3 implementation or that
in HttpCore. It will likely take me a few days to complete the analysis
and come up a fix.

Oleg


> On Tue, Oct 13, 2020 at 11:07 AM Oleg Kalnichevski <[email protected]>
> wrote:
> > On Tue, 2020-10-13 at 08:42 +0530, Arunan Sugunakumar wrote:
> > > Hi Oleg,
> > > 
> > > I upgraded to 4.4.13. Still the issue is there.
> > > 
> > 
> > Provide a reproducer.
> > 
> > Oleg
> > 
> > > Regards,
> > > Arunan
> > > 
> > > On Mon, Oct 12, 2020, 20:56 Oleg Kalnichevski <[email protected]>
> > > wrote:
> > > 
> > > > On Sun, 2020-10-11 at 22:16 +0530, Arunan Sugunakumar wrote:
> > > > > Hi,
> > > > > 
> > > > > We have used httpcore-NIO to implement a non-blocking server
> > and
> > > > > with
> > > > > a recent JAVA upgrade, we are finding a high CPU usage in our
> > > > > servers. While we debugged, we found out that when we close
> > the
> > > > > HTTPS
> > > > > connection on the server-side, the WRITE event is getting
> > > > > fired continuously and the connection close is unsuccessful.
> > > > > 
> > > > > To explain the code, we have extended
> > DefaultNHttpConnectionBase
> > > > > with
> > > > > our own ServerConnection, and inside close() we simply just
> > call
> > > > > the
> > > > > close() method of the NHttpConnectionBase[1]. If we look at
> > the
> > > > > code
> > > > > below, the close method sets the status as CLOSING and after
> > that
> > > > > either sets the WRITE event in the session or tries to close
> > the
> > > > > session (SSLIOSession in our case).
> > > > > 
> > > > > @Override
> > > > > public void close() throws IOException {
> > > > >     if (this.status != ACTIVE) {
> > > > >         return;
> > > > >     }
> > > > >     this.status = CLOSING;
> > > > >     if (this.outbuf.hasData()) {
> > > > >         this.session.setEvent(EventMask.WRITE);
> > > > >     } else {
> > > > >         this.session.close();
> > > > >         this.status = CLOSED;
> > > > >     }
> > > > > }
> > > > > 
> > > > > Inside SSLIOSession close(), the updateEventMask() [2] is
> > called.
> > > > > @Override
> > > > > public synchronized void close() {
> > > > >     if (this.status >= CLOSING) {
> > > > >         return;
> > > > >     }
> > > > >     this.status = CLOSING;
> > > > >     if (this.session.getSocketTimeout() == 0) {
> > > > >         this.session.setSocketTimeout(1000);
> > > > >     }
> > > > >     try {
> > > > >         updateEventMask();
> > > > >     } catch (final CancelledKeyException ex) {
> > > > >         shutdown();
> > > > >     }
> > > > > }
> > > > > 
> > > > > Inside updateEventMask() method, we get the HandshakeStatus
> > from
> > > > > the
> > > > > JAVA SSLEngine, and set a new Mask based on the status.
> > > > > switch (this.sslEngine.getHandshakeStatus()) {
> > > > > case NEED_WRAP:
> > > > >     newMask = EventMask.READ_WRITE;
> > > > >     break;
> > > > > case NEED_UNWRAP:
> > > > >     newMask = EventMask.READ;
> > > > >     break;
> > > > > case NOT_HANDSHAKING:
> > > > >     newMask = this.appEventMask;
> > > > >     break;
> > > > > case NEED_TASK:
> > > > >     break;
> > > > > case FINISHED:
> > > > >     break;
> > > > > }
> > > > > Before JDK upgrade, when outbound is closed, we used to get
> > the
> > > > > status as NEED_UNWRAP which sets the newMask as READ mask
> > which
> > > > > closes the connection in our server perfectly.
> > > > > But with a recent JDK commit[3], the new status is returned
> > as
> > > > > NOT_HANDSHAKING.
> > > > > Since mask is not set to READ because of the NOT_HANDSHAKING
> > > > > status,
> > > > > WRITE event is continuously getting fired which makes the
> > Server
> > > > > connection close unsuccessful.
> > > > > When the close is called multiple times for multiple
> > connections,
> > > > > we
> > > > > could observe that CPU usage increased continuosly and
> > reached
> > > > > 100
> > > > > percent and stays there continuously.
> > > > > When I look at the stack trace, the execution flow was
> > completely
> > > > > inside the httpcore-nio component, which made our attempt to
> > > > > clear
> > > > > the Event from the server implementation, unsuccessful.
> > > > > 
> > > > > 
> > > > > Earlier we were using Httpcore-NIO version 4.3.3. With this
> > > > > issue, we
> > > > > tried upgrading to 4.4.13. But still the issue exists.
> > > > 
> > > > I have seen no evidence supporting that. Please upgrade to
> > 4.4.13.
> > > > 
> > > > Oleg
> > > > 
> > > > 
> > > > 
> > > > -------------------------------------------------------------
> > ----
> > > > ----
> > > > To unsubscribe, e-mail: [email protected]
> > > > For additional commands, e-mail: [email protected]
> > > > 
> > > > 
> > 
> > 
> > -----------------------------------------------------------------
> > ----
> > To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail: [email protected]
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to