[ https://issues.apache.org/jira/browse/AVRO-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vaibhav updated AVRO-2888: -------------------------- Description: Csharp specific reader fails when csharp language keyword is present in namespace and datatype for on one of the element is array of complex type. Steps to reproduce - Use following schema file to generate avro csharp classes using Apache.Avro.Tools tool (v 1.10.0) {code:java} { "name": "Parent", "type": "record", "namespace": "com.parent.event", "fields": [ { "name": "children", "type": { "type": "array", "items": { "name": "Child", "type": "record", "fields": [ { "name": "name", "type": "string" } ] } } } ] } {code} Following classes will be generated. I have added link to files as content is relatively large. [https://github.com/vkhose/testcode/blob/master/com/parent/event/Parent.cs] [https://github.com/vkhose/testcode/blob/master/com/parent/event/Child.cs] Please note that Avro generator tool adds an escape character "@" in namespace as event is a csharp keyword e.g. namespace com.parent.@event Next, I have created a test case which tries to deserialize the Parent class object [https://github.com/vkhose/testcode/blob/master/testCaseToReproduceError.cs] {code:java} [Test] public void TestNamespaceWithCSharpKeyword() { var srcRecord = new Parent { children = new List<Child> { new Child { name = "test" }, new Child { name = "test" } } }; var stream = serialize(Parent._SCHEMA, srcRecord); var dstRecord = deserialize<Parent>(stream, Parent._SCHEMA, Parent._SCHEMA); Assert.NotNull(dstRecord); } }{code} This test case throws following exception Unable to find type 'com.parent.@event.Child' in all loaded assemblies in field children ----> Avro.AvroException : Unable to find type 'com.parent.@event.Child' in all loaded assemblies at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema writerSchema, Schema readerSchema, Decoder dec) When I debug the code it seems like same util methods are used to generate code for specific record and read record during deserialization. [https://github.com/apache/avro/blob/c0094b5bb1abb79304ce42a56cc115186370d407/lang/csharp/src/apache/main/CodeGen/CodeGenUtil.cs#L99] {code:java} if (ReservedKeywords.Contains(names[i])) builder.Append(At); {code} Appending At(@) while generating code is correct but it should not appended while deserializing. Deserialization fails as c# reflection does not expects At(@) in string value provided to create class instance. [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Specific/ObjectCreator.cs#L165] Type.GetType(string) string parameter passed should be without "@" character in namespace name. It works if I modify code not to add "@" in CodeGenUtil.cs. I think we need a fix where "@" is only added while auto generating classes and should not be used during deserialization. Thanks. Vaibhav was: Csharp specific reader fails when csharp language keyword is present in namespace and datatype for on one of the element is array of complex type. Steps to reproduce - Use following schema file to generate avro csharp classes using Apache.Avro.Tools tool (v 1.10.0) {code:java} { "name": "Parent", "type": "record", "namespace": "com.parent.event", "fields": [ { "name": "children", "type": { "type": "array", "items": { "name": "Child", "type": "record", "fields": [ { "name": "name", "type": "string" } ] } } } ] } {code} Following classes will be generated [Parent.cs|[https://github.com/vkhose/testcode/blob/master/com/parent/event/Parent.cs]] [Child.cs|[https://github.com/vkhose/testcode/blob/master/com/parent/event/Child.cs]] Please note that Avro generator tool adds an escape character "@" in namespace as event is a csharp keyword e.g. namespace com.parent.@event Next, I have created a test case which tries to deserialize the Parent class object [Test case code|[https://github.com/vkhose/testcode/blob/master/testCaseToReproduceError.cs]] {code:java} [Test] public void TestNamespaceWithCSharpKeyword() { var srcRecord = new Parent { children = new List<Child> { new Child { name = "test" }, new Child { name = "test" } } }; var stream = serialize(Parent._SCHEMA, srcRecord); var dstRecord = deserialize<Parent>(stream, Parent._SCHEMA, Parent._SCHEMA); Assert.NotNull(dstRecord); } }{code} This test case throws following exception Unable to find type 'com.parent.@event.Child' in all loaded assemblies in field children ----> Avro.AvroException : Unable to find type 'com.parent.@event.Child' in all loaded assemblies at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema writerSchema, Schema readerSchema, Decoder dec) When I debug the code it seems like same util methods are used to generate code for specific record and read record during deserialization. [https://github.com/apache/avro/blob/c0094b5bb1abb79304ce42a56cc115186370d407/lang/csharp/src/apache/main/CodeGen/CodeGenUtil.cs#L99] {code:java} if (ReservedKeywords.Contains(names[i])) builder.Append(At); {code} Appending At(@) while generating code is correct but it should not appended while deserializing. Deserialization fails as c# reflection does not expects At(@) in string value provided to create class instance. [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Specific/ObjectCreator.cs#L165] Type.GetType(string) string parameter passed should be without "@" character in namespace name. It works if I modify code not to add "@" in CodeGenUtil.cs. I think we need a fix where "@" is only added while auto generating classes and should not be used during deserialization. Thanks. Vaibhav > Csharp - Specific reader fails when using c# keyword in namespace name > ---------------------------------------------------------------------- > > Key: AVRO-2888 > URL: https://issues.apache.org/jira/browse/AVRO-2888 > Project: Apache Avro > Issue Type: Bug > Components: csharp > Affects Versions: 1.9.2 > Reporter: Vaibhav > Priority: Major > > Csharp specific reader fails when csharp language keyword is present in > namespace and datatype for on one of the element is array of complex type. > Steps to reproduce - Use following schema file to generate avro csharp > classes using Apache.Avro.Tools tool (v 1.10.0) > {code:java} > { > "name": "Parent", > "type": "record", > "namespace": "com.parent.event", > "fields": [ > { > "name": "children", > "type": { > "type": "array", > "items": { > "name": "Child", > "type": "record", > "fields": [ > { > "name": "name", > "type": "string" > } > ] > } > } > } > ] > } > {code} > Following classes will be generated. I have added link to files as content is > relatively large. > [https://github.com/vkhose/testcode/blob/master/com/parent/event/Parent.cs] > [https://github.com/vkhose/testcode/blob/master/com/parent/event/Child.cs] > Please note that Avro generator tool adds an escape character "@" in > namespace as event is a csharp keyword > e.g. namespace com.parent.@event > Next, I have created a test case which tries to deserialize the Parent class > object > [https://github.com/vkhose/testcode/blob/master/testCaseToReproduceError.cs] > > {code:java} > [Test] > public void TestNamespaceWithCSharpKeyword() > { > var srcRecord = new Parent > { > children = new List<Child> > { > new Child > { > name = "test" > }, > new Child > { > name = "test" > } > } > }; > var stream = serialize(Parent._SCHEMA, srcRecord); > var dstRecord = deserialize<Parent>(stream, > Parent._SCHEMA, Parent._SCHEMA); > Assert.NotNull(dstRecord); > } > }{code} > This test case throws following exception > Unable to find type 'com.parent.@event.Child' in all loaded assemblies in > field children > ----> Avro.AvroException : Unable to find type 'com.parent.@event.Child' in > all loaded assemblies > at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema > writerSchema, Schema readerSchema, Decoder dec) > > When I debug the code it seems like same util methods are used to generate > code for specific record and read record during deserialization. > > [https://github.com/apache/avro/blob/c0094b5bb1abb79304ce42a56cc115186370d407/lang/csharp/src/apache/main/CodeGen/CodeGenUtil.cs#L99] > {code:java} > if (ReservedKeywords.Contains(names[i])) > builder.Append(At); > {code} > Appending At(@) while generating code is correct but it should not appended > while deserializing. Deserialization fails as c# reflection does not expects > At(@) in string value provided to create class instance. > > [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Specific/ObjectCreator.cs#L165] > Type.GetType(string) string parameter passed should be without "@" character > in namespace name. > > It works if I modify code not to add "@" in CodeGenUtil.cs. I think we need a > fix where "@" is only added while auto generating classes and should not be > used during deserialization. > Thanks. > Vaibhav -- This message was sent by Atlassian Jira (v8.3.4#803005)