[
https://issues.apache.org/jira/browse/KAFKA-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17380772#comment-17380772
]
Konstantine Karantasis commented on KAFKA-12308:
------------------------------------------------
Adding the comment that I added in the PR here as well:
The idea that the {{DelegatingClassLoader}} did not have to be parallel capable
originated to the fact that it doesn't load classes directly. It delegates
loading either to the appropriate PluginClassLoader directly via composition,
or to the parent by calling {{super.loadClass}}.
The latter is the key point of why we need to make the
{{DelegatingClassLoader}} also parallel capable even though it doesn't load a
class. Because inheritance is used (via a call to {{super.loadClass}}) and not
composition (via a hypothetical call to {{parent.loadClass}}, which is not
possible because {{parent}} is a private member of the base abstract class
{{ClassLoader}}) when {{getClassLoadingLock}} is called in {{super.loadClass}}
it checks that actually the derived class (here an instance of
{{DelegatingClassLoader}}) is not parallel capable and therefore ends up not
applying fine-grain locking during classloading even though the parent
clasloader is used actually load the classes.
Based on the above, the {{DelegatingClassLoader}} needs to be parallel capable
too in order for the parent loader to load classes in parallel.
I've tested both classloader types being parallel capable in a variety of
scenarios with multiple connectors, SMTs and converters and a deadlock did not
reproduce. Of course reproducing the issue is difficult without the specifics
of the jar layout to begin with. The possibility of a deadlock is still not
zero, but also probably not exacerbated compared to the current code. The
plugin that depends on other plugins to be loaded while it's loading its
classes is the connector type plugin only and there are no inter-connector
dependencies (a connector requiring another connector's classes to be loaded
while loading its own). With that in mind, a deadlock should be even less
possible now. In the future we could consider introducing deadlock recovery
methods to get out of this type of situation if necessary.
> ConfigDef.parseType deadlock
> ----------------------------
>
> Key: KAFKA-12308
> URL: https://issues.apache.org/jira/browse/KAFKA-12308
> Project: Kafka
> Issue Type: Bug
> Components: config, KafkaConnect
> Affects Versions: 2.5.0
> Environment: kafka 2.5.0
> centos7
> java version "1.8.0_231"
> Reporter: cosmozhu
> Priority: Major
> Attachments: deadlock.log
>
>
> hi,
> the problem was found, when I restarted *ConnectDistributed*
> I restart ConnectDistributed in the single node for the test, with not delete
> connectors.
> sometimes the process stopped when creating connectors.
> I add some logger and found it had a deadlock in `ConfigDef.parseType`.My
> connectors always have the same transforms. I guess when connector startup
> (in startAndStopExecutor which default 8 threads) and load the same class
> file it has something wrong.
> I attached the jstack log file.
> thanks for any help.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
