Re: Pig to Hadoop complex data exchange format

2011-05-25 Thread Dmitriy Ryaboy
You can use Thrift or Protobufs using elephant-bird. D On Wed, May 25, 2011 at 10:29 AM, Andrew Hammond wrote: > Bump. I'm very interested in combining Avro and Pig and would greatly > appreciate hearing about people's experiences with them. > > On Mon, Apr 18, 2011 at 2:23 AM, Gianmarco wrote:

Re: Pig to Hadoop complex data exchange format

2011-05-25 Thread Andrew Hammond
Bump. I'm very interested in combining Avro and Pig and would greatly appreciate hearing about people's experiences with them. On Mon, Apr 18, 2011 at 2:23 AM, Gianmarco wrote: > Last time I checked Avro on MR the integration was not yet ready. > > I see that there was a very recent release of c

Re: Pig to Hadoop complex data exchange format

2011-04-18 Thread Gianmarco
Last time I checked Avro on MR the integration was not yet ready. I see that there was a very recent release of code for it http://www.tomslabs.com/ but I don't know how stable and tested this code is. Has anyone had good experiences with Avro on MR? Cheers, -- Gianmarco De Francisci Morales O

Re: Pig to Hadoop complex data exchange format

2011-04-15 Thread Harsh J
Hey Gianmarco, On Fri, Apr 15, 2011 at 5:00 PM, Gianmarco wrote: > Avro is not an easy option on the hadoop side. Am just a little curious on this, could you explain why you feel so about Avro on M/R? -- Harsh J

Pig to Hadoop complex data exchange format

2011-04-15 Thread Gianmarco
Hi all, I have a pig script that produces a complex nested data structure: result: {child: chararray, childTraces: {action: int, time: long}, legacy: {parent: chararray, parentTraces: {action: int, time: long}}} I would like to post-process the output of the pig script with a mapreduce job. In t