[
https://issues.apache.org/jira/browse/AVRO-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599973#comment-14599973
]
Ryan Blue commented on AVRO-1687:
---------------------------------
Sorry, I just commented on this problem on AVRO-1604, following the order in my
inbox. Anyway, I think the problem is that your union isn't encoded according
to the [Avro spec for
JSON|https://avro.apache.org/docs/1.7.7/spec.html#json_encoding]. If you create
the record by hand and then convert it to JSON, you'll see an extra layer for
the union that tells Avro which union type the JSON record is.
There are other utilities that can convert from raw JSON, as you're trying to
do here. I recommend taking a look at Kite. There's also a ticket to add a
special case to Avro for nullable types, like this. But that is a new JSON
encoding needs someone to push it forward.
> Converting JSON to avro fails
> ------------------------------
>
> Key: AVRO-1687
> URL: https://issues.apache.org/jira/browse/AVRO-1687
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.7.6, 1.7.7, 1.8.0
> Reporter: Dean Ericson
>
> I am trying to validate a JSON string that is sent to me. Unfortunately, I
> get a stacktrace indicating that my schema and JSON are not compatable:
> {code}org.apache.avro.AvroTypeException: Unknown union branch hour{code} This
> does not seem right to me since the JSON being passed to me was created from
> the same schema I am using to validate it.
> Here are my tests
> {code}
> @BeforeClass
> public static void testGenerateSchema() throws IOException {
> /************************************************
> * GENERATE SCHEMA
> ************************************************/
> // get the reflected schema for packets
> File schemaFile = new File("avro.schema");
> schema = ReflectData.AllowNull.get().getSchema(Packet.class);
> try (DataFileWriter<Schema> schemaWiter = new DataFileWriter<>(
> new GenericDatumWriter<Schema>())) {
> schemaWiter.create(schema, schemaFile);
> schemaWiter.close();
> }
> assertNotNull(schemaFile);
> assertTrue(schemaFile.length() > 0);
> }
> @Test
> public void testAvroJsonToBinarySerialization() throws Exception {
> /************************************************
> * SERIALIZE JSON
> ************************************************/
> String json = "{\"cost\": 0, \"stamp\": {\"hour\": 12,
> \"second\": 0}}";
> Schema schema = new
> Schema.Parser().parse(getSchema().toString());
> System.out.println(schema);
> System.out.println(jsonToAvro(json, schema));
> }
> // Code borrowed from the Avro-Tools project
> private static String jsonToAvro(String json, Schema schema)
> throws IOException {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> PrintStream out = new PrintStream(new
> BufferedOutputStream(baos));
> GenericDatumReader<Object> reader = new
> GenericDatumReader<Object>(schema);
> InputStream input = new
> ByteArrayInputStream(json.getBytes("utf-8"));
> JsonDecoder jsonDecoder =
> DecoderFactory.get().jsonDecoder(schema,input);
> GenericDatumWriter<Object> writer = new
> GenericDatumWriter<Object>(schema);
> Encoder e = EncoderFactory.get().binaryEncoder(out, null);
> Object datum = null;
> while (true) {
> try {
> datum = reader.read(datum, jsonDecoder);
> } catch (EOFException eofException) {
> break;
> }
> writer.write(datum, e);
> e.flush();
> }
> return baos.toString("utf-8");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
