gharris1727 commented on PR #14064: URL: https://github.com/apache/kafka/pull/14064#issuecomment-1661121133
> Hmmm... getting an NPE when I try to run the command now: I've fixed that, sorry. I've also added tests that would catch that NPE. > made a note of any collisions that we detected in the summary section This is a better solution than hiding it in the row listing, as the collisions are much more apparent. I've ported the DEBUG log contents to stdout, and made it part of the summary. See the new summary below: ``` Ignoring ambiguous alias 'ByteArray' since it refers to multiple distinct plugins [org.apache.kafka.connect.converters.ByteArrayConverter, io.debezium.converters.ByteArrayConverter, io.confluent.connect.cloud.storage.source.format.ByteArrayConverter, io.confluent.connect.replicator.util.ByteArrayConverter] <snipped> Ignoring ambiguous alias 'ByteArrayConverter' since it refers to multiple distinct plugins [org.apache.kafka.connect.converters.ByteArrayConverter, io.debezium.converters.ByteArrayConverter, io.confluent.connect.cloud.storage.source.format.ByteArrayConverter, io.confluent.connect.replicator.util.ByteArrayConverter] Total plugins: 419 Loadable plugins: 419 Compatible plugins: 7 Total locations: 150 Compatible locations: 10 All locations compatible? false ``` I had to duplicate some of the alias computation logic, since the PluginUtils implementation is tightly coupled to the PluginDesc objects, which we can't use in the summary. If we keep the PluginScanResults from the scanning portion, we end up leaking enough memory that the default 256MB heap runs out of space. Now the script is incremental (again) and can iterate over multi-gb plugin paths with a bounded heap, less than the corresponding worker at least (which gets a 2GB heap by default). Unfortunately, if there are JDBC drivers included in a plugin, then it will leak memory until the script exits. I don't expect this will be an issue in practice (i'm already testing plugins including JDBC drivers), and people can always increase their heap to compensate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
