Gerald Schermann created AVRO-4357:
--------------------------------------
Summary: [C#] ObjectCreator enumerates all types of all loaded
assemblies for every schema name
Key: AVRO-4357
URL: https://issues.apache.org/jira/browse/AVRO-4357
Project: Apache Avro
Issue Type: Improvement
Components: csharp
Affects Versions: 1.12.2
Reporter: Gerald Schermann
{{ObjectCreator.FindType}} resolves the generated class of a named schema type
by name. If the class is not in the entry assembly, the usual case for
generated contracts, it enumerates all types of all loaded assemblies, without
stopping at a match and allocating an unmangled copy of the name per type. This
runs once per named schema type and process, so the first deserialization of a
schema costs (named types in the schema) x (loaded types).
We hit this in a large .NET 10 service consuming Kafka with Confluent.Kafka
2.15.0 and {{AvroDeserializer<T>}} (Confluent.SchemaRegistry.Serdes.Avro
2.15.0) on Apache.Avro 1.12.2: the first message after every process start took
about 5 seconds to deserialize, the following ones milliseconds.
We benchmarked time and allocations to resolve (the generated) classes of 50
named schema types with an empty type cache (a new {{{}ObjectCreator{}}}),
depending on how many types the process has loaded (BenchmarkDotNet, .NET 10,
Apple M2 Max):
||Types loaded in the process||Time to resolve all 50||Allocated||
|5,938|286.9 ms|292 MB|
|30,938|1,017.6 ms|940 MB|
|105,938|2,720.9 ms|2,708 MB|
|255,938|5,970.5 ms|6,244 MB|
h3. Proposed fix
Look up names with a namespace directly with {{Assembly.GetType(name, false)}}
in each loaded assembly, and keep the existing enumeration as fallback. This
resolves the same types as before, including the last match winning across
assemblies, and brings the 255,938-type case down from 5,970.5 ms and 6,244 MB
allocated to 4.4 ms and 4.5 MB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
