Dynamic schema resolution from writer to reader
-----------------------------------------------

                 Key: AVRO-220
                 URL: https://issues.apache.org/jira/browse/AVRO-220
             Project: Avro
          Issue Type: Improvement
          Components: c++
            Reporter: Scott Banachowski


This introduces a new feature for C++, which enables data written by one schema 
to be read by a different schema.

Before parse time, it examines the two schemas, and assembles a list of 
instructions on how to parse the data, and how to translate the parsed data to 
the expected schema (or skip it, as appropriate).  The result of this operation 
is a sequence of instructions.

At parse time, it basically follows the assembled list of instructions.  The 
instructions can be created once, and then reused to read multiple messages 
(analogous to assembling a program that does the parsing, and then executing 
this program when you want to read the data).

A structure is used when assembling the instructions, that holds a mapping of 
memory locations for primitives (relative to the start of some structure).   
This tells the parser where to copy the data to while parsing from the writer 
schema to a C++ structure. For non-primitive types, it stores the memory 
location for a functor that returns a destination address when invoked (for 
example, to add a map entry the functor will allocate the map entry, and return 
an address for the parser to write the actual data.  Then the instruction for 
the map data type is invoked to populate this address).  Setting up this offset 
structure is a chore, but using the code generator make it simple. The code 
generator correctly cranks out these structures for all the data types it 
generates. 




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to