Randall Hauch created KAFKA-6168:
------------------------------------
Summary: Connect Schema comparison is slow for large schemas
Key: KAFKA-6168
URL: https://issues.apache.org/jira/browse/KAFKA-6168
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Affects Versions: 1.0.0
Reporter: Randall Hauch
Priority: Critical
The {{ConnectSchema}} implementation computes the hash code every time its
needed, and {{equals(Object)}} is a deep equality check. This extra work can be
expensive for large schemas, especially in code like the {{AvroConverter}} (or
rather {{AvroData}} in the converter) that uses instances as keys in a hash map
that then requires significant use of {{hashCode}} and {{equals}}.
The {{ConnectSchema}} is an immutable object and should at a minimum precompute
the hash code. Also, the order that the fields are compared in {{equals(...)}}
should use the cheapest comparisons first (e.g., the {{name}} field is one of
the _last_ fields to be checked). Finally, it might be worth considering having
each instance precompute and cache a string or byte[] representation of all
fields that can be used for faster equality checking.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
