Hi all,

*JIRA*: https://issues.apache.org/jira/browse/APEXMALHAR-2034
*Pull Request*: https://github.com/apache/apex-malhar/pull/666

*Functional Change*
As per the above JIRA, we have created a new Avro Module to address the
following issue

*Issue:*
Avro objects are not serialized by Kryo causing the Avro GenericRecord to
not be available to downstream operators if users don't explicitly mark the
stream locality at container_local or thread_local.

*Solution:*
We have created a Module on top of AvroFileInputOperator and AvroToPojo
operators such that downstream operators will access POJO instead of Avro
GenericRecord. It, therefore, removes the exposure of GenericRecord to
downstream operators and instead exposes the created POJO to downstream
operators.

In this Module, the stream between the two encapsulated operators
(AvroFileInputOperator and AvroToPojo) is set to CONTAINER_LOCAL.


*Avro package move out of contrib*

Along with creating new avro module, existing avro files are moved from
contrib module to a new 'avro' package under malhar repo.


   - This move is in accordance to JIRA
   https://issues.apache.org/jira/browse/APEXMALHAR-1843
   - To ensure backward compatibility, old operator files are marked
   deprecated and made to extend from new operator files.
   - Git history of all the moved files is maintained

Unit/application-level tests done are elaborated in the JIRA.

Please let me know if any concerns around this dev work.


Best Regards,

Saumya Mohan

Reply via email to