[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103165#comment-17103165
]
Fokko Driesprong commented on AVRO-2805:
Thanks, [~pcless] for sharing this with us, and [~anhldbk] for looking into the
issue. I'll close this one for now.
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> public Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17087023#comment-17087023
]
Andy Le commented on AVRO-2805:
---
[~pcless] Yep. Be strong!
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> public Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086921#comment-17086921
]
Pedro Cardoso Silva commented on AVRO-2805:
---
I can not reproduce the issue now :D
For curiosity, this is the full test code (the only difference to my original
case was that the enum and class being serialized are in separate files):
{code:java}
import io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient
import io.confluent.kafka.schemaregistry.client.SchemaRegistryClient
import io.confluent.kafka.serializers.KafkaAvroDeserializer
import io.confluent.kafka.serializers.KafkaAvroDeserializerConfig
import io.confluent.kafka.serializers.KafkaAvroSerializer
import org.apache.avro.reflect.ReflectData
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test
import java.util.HashMap
class ReflectionAvroSerDeTest {
private val schemaRegistry: SchemaRegistryClient =
MockSchemaRegistryClient()
private val reflectionAvroDeserializer: KafkaAvroDeserializer
private val reflectionAvroSerializer: KafkaAvroSerializer
private val topic = "test";
init {
val reflectionProps = HashMap()
// Intentionally invalid schema registry URL to satisfy the config
class's requirement that
// it be set.
// Intentionally invalid schema registry URL to satisfy the config
class's requirement that
// it be set.
reflectionProps[KafkaAvroDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG]
= "bogus"
reflectionProps[KafkaAvroDeserializerConfig.SCHEMA_REFLECTION_CONFIG] =
"true"
reflectionAvroSerializer = KafkaAvroSerializer(schemaRegistry,
reflectionProps)
reflectionAvroDeserializer = KafkaAvroDeserializer(schemaRegistry,
reflectionProps)
}
@Test
fun definitionTest() {
val definition = Definition(mapOf("a" to Type.A))
val avroSchema = ReflectData.get().getSchema(Definition::class.java)
val bytes = reflectionAvroSerializer.serialize(topic, definition)
val obj = reflectionAvroDeserializer.deserialize(topic, bytes,
avroSchema)
assertEquals(
Definition::class.java,
obj.javaClass,
"Returned object should be a Definition type"
)
assertEquals(definition, obj)
}
class Definition(val tokens: Map = mapOf()) {
}
enum class Type {
A, B, C
}
}
{code}
I am now wondering whether the original issue was not related to Avro but to
Kafka's Avro Serialization logic.
Either way, thank you for your time, Andy.
Have a great weekend.
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> public Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086916#comment-17086916
]
Andy Le commented on AVRO-2805:
---
[~pcless] I'm just a contributor :-P Can't close your issue.
But feel free to show your Kotlin snippet. I think there's nothing specific to
Kotlin here.
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> public Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086913#comment-17086913
]
Pedro Cardoso Silva commented on AVRO-2805:
---
The original code is in Kotlin as a data class & enum class, it seems I can not
reproduce the issue in Java sample test.
I will investigate further, you may close this issue in the meantime.
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> public Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2805) Reflection-based schema not loading types
[
https://issues.apache.org/jira/browse/AVRO-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086869#comment-17086869
]
Andy Le commented on AVRO-2805:
---
[~pcless] Your example code contain errors. I decided to wrap your logic inside
a test
{code:java}
public static class Definition {
public Map tokens;
}
public enum Type {
A,
B,
C
}
@Test
public void testAvro2805() {
Schema schema = ReflectData.get().getSchema(Definition.class);
final String schemaString = schema.toString(true);
System.out.println(schemaString);
}
{code}
I can NOT see your exception above. I've got this one instead:
{noformat}
org.apache.avro.SchemaParseException: Illegal character in: this$0
at org.apache.avro.Schema.validateName(Schema.java:1530)
at org.apache.avro.Schema.access$400(Schema.java:87)
at org.apache.avro.Schema$Field.(Schema.java:518)
at org.apache.avro.Schema$Field.(Schema.java:557)
at
org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:683)
{noformat}
Please kindly verify your issue again and let me know.
> Reflection-based schema not loading types
> --
>
> Key: AVRO-2805
> URL: https://issues.apache.org/jira/browse/AVRO-2805
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.9.1
>Reporter: Pedro Cardoso Silva
>Priority: Critical
>
> Avro reflection is unable to generate a schema for the following definition:
> {code:java}
> public class Definition {
> pubilc Map
> }
> public enum Type {
> A,
> B,
> C
> }
> {code}
> {code:java}
> // Test code
> Schema schema = ReflectData.get().getSchema(Definition.class)
> {code}
> Fails with:
> Undefined name: "FieldType"
> org.apache.avro.SchemaParseException: Undefined name: "Type"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
