[ https://issues.apache.org/jira/browse/PIG-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-3111: ---------------------------- Fix Version/s: (was: 0.14.0) 0.15.0 > ToAvro to convert any Pig record to an Avro bytearray > ----------------------------------------------------- > > Key: PIG-3111 > URL: https://issues.apache.org/jira/browse/PIG-3111 > Project: Pig > Issue Type: New Feature > Components: data, internal-udfs > Affects Versions: 0.12.0 > Reporter: Russell Jurney > Assignee: Russell Jurney > Fix For: 0.15.0 > > > I want to create a ToAvro() builtin that converts arbitrary pig fields, > including complex types (bags, tuples, maps) to avro format as bytearrays. > This would enable storing Avro records in arbitrary data stores, for example > HBaseAvroStorage in PIG-2889 > See PIG-2641 for ToJson > This points to a greater need for customizable/pluggable serialization that > plugin to storefuncs and do serialization independently. For example, we > might do these operations: > a = load 'my_data' as (some_schema); > b = foreach a generate ToJson(*); > c = foreach a generate ToAvro(*); > store b into 'hbase://JsonValueTable' using HBaseStorage(...); > store c into 'hbase://AvroValueTable' using HBaseStorage(...); > I'll make a ticket for pluggable serialization separately. -- This message was sent by Atlassian JIRA (v6.3.4#6332)