[
https://issues.apache.org/jira/browse/CXF-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782448#comment-17782448
]
Lars Ködderitzsch commented on CXF-8951:
----------------------------------------
Sorry for the delay.
I retestet with 4.0.4-SNAPSHOT - the main issue is *not* fixed with this
version.
Only the runaway thread leak exposed in "clientPerRequestWithNewConduit" is not
present anymore.
However, thread usage in both
"singleClientInstanceWithNewConduit"/"clientPerRequestWithNewConduit" is still
huge in comparison to the similar tests with the old conduit.
If you would have a look at the attached reproducer project and its test output
this will immediately become apparent.
The issue at hand seems very much to be how the new conduits usage of the java
http client is implemented. It can't be quite right if every client/requestor
thread is using its own java http client instance - as the java http client
itself manages selector/worker threads.
As of now this leads to each java http client having its own short lived thread
pool which is extremely wasteful.
Instead it appears that having all clients sharing a single, properly
configured java http client would be more fitting, concept wise. The number of
worker threads should be configurable by cxf users in this case and should (of
course) have a sensible default - similar a servlet container having a sensible
default on worker threads for incoming requests.
> Concurrent WebClient usage causes massive thread overhead
> ---------------------------------------------------------
>
> Key: CXF-8951
> URL: https://issues.apache.org/jira/browse/CXF-8951
> Project: CXF
> Issue Type: Bug
> Components: Core, JAX-RS
> Affects Versions: 3.6.2, 4.0.3
> Reporter: Lars Ködderitzsch
> Priority: Major
> Attachments: cxf-connector-threads.zip
>
>
> Concurrent usage of Webclients causes massively more threads to be created in
> cxf-3.6.x/4.x than before.
> Supposedly this results from introducing a new conduit implementation based
> on the "new" java http client compared to HttpUrlConnection before.
> It seems that a new http client is created per requesting thread - and each
> http client in turn creates an own pool of selector/worker threads.
> To demonstate I've attached a test project with several methods to test
> different scenarios.
> Each tests launches a simple JAX-RS Server, configures/uses WebClients in
> different configuration and submits 1000 requests via up to 100 parrallel
> requestor threads.
> The number of live threads in the JVM instance if printed after each request.
> Baseline is the number of threads used by the server instance + the 100
> requesting threads.
>
> singleClientInstanceWithNewConduit -> reuses a threadsafe client instance,
> thread count rises to 700+ live threads
> singleClientInstanceWithOldConduit -> reuses a threadsafe client instance,
> stays at about 200 threads
> clientPerRequestWithOldConduit -> creates a client instance per request
> (no closing!), keeps below 200 threads
> clientPerRequestWithNewConduit -> creates a client instance per request
> (no closing!), creates a runaway thread leak (!)
> clientPerRequestWithNewConduit is additionally interesting because current
> documentation is not clear about whether client instances
> are required to be resource managed or not.
> It appears that closing clients was not required up until cxf-3.6.0 and
> documentation actively encourages creating new "lightweight" client instances
> on the fly (see Thread safety in
> [https://cxf.apache.org/docs/jax-rs-client-api.html]).
> However, cxf-3.6+ implementation (and comments in CXF-8885) seem to suggest
> that clients are in fact *not lightweight* (anymore?) but need to be closed,
> when no longer used.
> But then in turn WebClient/Client does not even implement
> Closeable/Autoclosable. So, it's all quite muddy - and there doesn't seem to
> be a real consensus on the idiomatic usage of Webclients.
> It would be very nice if you could bring some clarity on this as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
