Dynamic schema resolution from writer to reader
-----------------------------------------------
Key: AVRO-220
URL: https://issues.apache.org/jira/browse/AVRO-220
Project: Avro
Issue Type: Improvement
Components: c++
Reporter: Scott Banachowski
This introduces a new feature for C++, which enables data written by one schema
to be read by a different schema.
Before parse time, it examines the two schemas, and assembles a list of
instructions on how to parse the data, and how to translate the parsed data to
the expected schema (or skip it, as appropriate). The result of this operation
is a sequence of instructions.
At parse time, it basically follows the assembled list of instructions. The
instructions can be created once, and then reused to read multiple messages
(analogous to assembling a program that does the parsing, and then executing
this program when you want to read the data).
A structure is used when assembling the instructions, that holds a mapping of
memory locations for primitives (relative to the start of some structure).
This tells the parser where to copy the data to while parsing from the writer
schema to a C++ structure. For non-primitive types, it stores the memory
location for a functor that returns a destination address when invoked (for
example, to add a map entry the functor will allocate the map entry, and return
an address for the parser to write the actual data. Then the instruction for
the map data type is invoked to populate this address). Setting up this offset
structure is a chore, but using the code generator make it simple. The code
generator correctly cranks out these structures for all the data types it
generates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.